Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] decode error(#1806) #1809

Closed
wants to merge 1 commit into from

Conversation

mingchuno
Copy link

Short Description:

[Fix] decode error(#1806)

Fixes:

  • fix exception like "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 31: ordinal not in range(128)"

@gregtampa
Copy link
Collaborator

someone test and comment please

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

Not working for me.

$ python pokecli.py -cf config.json 
[10:20:29] PokemonGO Bot v1.0
[10:20:29] Configuration initialized
[10:20:29] 
[10:20:29] Not logged in!
Traceback (most recent call last):
  File "pokecli.py", line 464, in <module>
    main()
  File "pokecli.py", line 79, in main
    report_summary(bot)
  File "pokecli.py", line 87, in report_summary
    metrics.capture_stats()
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/metrics.py", line 75, in capture_stats
    response_dict = self.bot.api.call()
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 57, in call
    if not self._can_call():
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 29, in _can_call
    raise NotLoggedInException()
pgoapi.exceptions.NotLoggedInException

As soon as I make decoded_str = string instead of decoded_string = string.decode('utf-8') it starts again but the original problem is still there, of course.

@mingchuno
Copy link
Author

I am running it myself and it works
@vieira for your case, It looks like totally different issue from your stacktrace

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

The fix that works for me is:

--- a/pokemongo_bot/cell_workers/move_to_fort_worker.py
+++ b/pokemongo_bot/cell_workers/move_to_fort_worker.py
@@ -40,7 +40,8 @@ class MoveToFortWorker(object):
         )

         if dist > Constants.MAX_DISTANCE_FORT_IS_REACHABLE:
-            logger.log('Moving towards fort {}, {} left'.format(fort_name, format_dist(dist, unit))
+            logger.log(u'Moving towards fort {}, {} left'
+                    .format(fort_name.decode('utf-8'), format_dist(dist, unit)))

             step_walker = StepWalker(
                 self.bot,

@mingchuno Fell free the use the fix above in your PR if you see fit.

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@mingchuno It's the same problem, the Unicode exception is probably getting caught by a generic except that just says unable to login. It's probably because my first name as a special character 😉

@mingchuno
Copy link
Author

@vieira We should solve it in lower layer since the logger is shared by the whole app.
Adding .decode('utf-8') inside the logger impl is better than adding .decode('utf-8') every where in the code

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@mingchuno Yes, I agree but in that case you have the fix every call to the logger that are already passing utf-8 encoded strings. Forget about my first name, it's probably the initial location.

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@mingchuno
Copy link
Author

@vieira Well, I am not actually know much on Python.
Isn't that u'test'.decode('utf-8') still give a valid unicode string?

@Di9
Copy link

Di9 commented Jul 30, 2016

it works for me fine, thx! But Instead of Cyrillic title i got in terminal pokestot name with wrong encoding anyway (before it also was with not-cyrilic encoding, just another one) :)

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@mingchuno You encode a sequence of unicode characters into a str, e.g. print(u'ç'.encode('utf-8')). u'ç'.decode('utf-8') won't work. You use decode, for instance, to bring a some encoded str back to unicode, you have to know how the str was encoded and decode it with the same codec, e.g. 'utf-8'. I am no expert in this, either.

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@mingchuno Can you reproduce my problem if you set your initial location to something with special chars, e.g. "ç" or "ü"?

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@mingchuno Without changing anything else, I think this works:

def log(string, color='white'):
    try:
        string = string.decode('utf8')
    except UnicodeEncodeError:
        pass

<rest of the original code>

Sometimes log is called with an already decoded str, other with an unicode sequence. If it is already a string decode will fail but it is safe to continue.

@sgitkene
Copy link

sgitkene commented Jul 30, 2016

@vieira your fix seems to do it for me: (I used to have an error due to the "ü" in Sihlbrücke)

[12:31:10] PokeStop on cooldown. Time left: 4 minutes, 59 seconds
[12:31:13] Moving towards fort 1866 Sihlbrücke, 0.11km left
[12:31:19] -- Recycled 3x Pokeball (keeps only 50 maximum)
[12:31:20] -- Recycled 1x Super Potion (keeps only 1 maximum)
[12:31:20] Moving towards fort 1866 Sihlbrücke, 0.10km left
[12:31:25] Moving towards fort 1866 Sihlbrücke, 0.08km left
[12:31:31] Something rustles nearby!

Edit: Apparently somethng's still wrong: (Why do we use ascii???)

Traceback (most recent call last):
File "pokecli.py", line 464, in
main()
File "pokecli.py", line 68, in main
bot.tick()
File "/home/max/PokemonGo-Bot/pokemongo_bot/init.py", line 84, in tick
if worker.work() == WorkerResult.RUNNING:
File "/home/max/PokemonGo-Bot/pokemongo_bot/cell_workers/seen_fort_worker.py", line 32, in work
logger.log('Now at Pokestop: {0}'.format(fort_name), 'cyan')
File "/home/max/PokemonGo-Bot/pokemongo_bot/logger.py", line 28, in log
print('[' + time.strftime("%H:%M:%S") + '] ' + u'\033[' + color_hex[color] + string.decode('utf-8') + '\033[0m')
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 28: ordinal not in range(128)

Edit2:

I put @CapCap 's fix in too and now it works. I believe nowhere should we still use ascii.
reload(sys)
sys.setdefaultencoding('utf8')

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@sgitkene Ooops, missed an u'' in line 28. I will open a PR with both fixes.

Should be working now (Now at Pokestop: Quiosque Cais do Sodré):

[11:45:01] Moving towards fort Quiosque Cais do Sodré , 0.05km left
[11:45:06] Moving towards fort Quiosque Cais do Sodré , 0.04km left
[11:45:11] Now at Pokestop: Quiosque Cais do Sodré 
[11:45:11] Spinning ...

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@sgitkene Could you help testing #1824? Thanks!

@WebberTakkn
Copy link

Please approve

@DayBr3ak
Copy link
Contributor

190640f fixed this

@sgitkene
Copy link

sorry @vieira I was away. I assume it is no longer needed?

@vieira
Copy link
Contributor

vieira commented Jul 30, 2016

@sgitkene No problem at all! I guess #1839 should have fixed this but I can't test right now. If it works with:

  1. initial location with special characters
  2. moving to fort with special characters
  3. arrived at pokestop with special characters

then we can close #1824.

@marksweb
Copy link
Contributor

marksweb commented Aug 1, 2016

@vieira I've come up against a couple more UnicodeDecode errors with some French place names. It seems that setting the encoding when creating the string passed to logger.log() proves a better solution, e.g. 'Now at Pokestop: {0}'.format(unicode(fort_name, 'utf-8'))

I've done this with spin_fort.py and move_to_fort.py & had it running for an hour or so with no issues. Maybe it's worth another PR?

@elicwhite
Copy link
Contributor

The logger has been deprecated and replaced with websockets. Hopefully this isn't an issue anymore. If it is, please re-open against the new approach.

@elicwhite elicwhite closed this Aug 3, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants